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Schools and universities all over the world are continuously exploring ways to use the 
web technology in improving teaching effectiveness. The use of course web pages, 
discussion groups, bulletin boards, and e-mails have shown impact on teaching and 
learning in significant ways, across all disciplines. e-Learning has emerged as an 
alternative to traditional classroom-based education and training, especially for 
distance learning programs. Thus, this research study was conducted to investigate 
the effectiveness of the implementation of webcasting technology in teaching in higher 
education. In this research, three modes of webcasting lectures were experimented on 
three different groups of students, using the pre test-post test-control group 
experimental design. The modes are live streaming, pre-recorded streaming and video 
on demand (VOD). The group that attended the face-to-face lecture acts as the con trol 
group. The overall analysis showed that the students who went through the VOD 
group showed the most gain in the tests. 

Webcasting technology, online learning, web-based learning, distance learning, higher education 


INTRODUCTION 

Computer-based systems have great potential for delivering teaching and learning material. 
According to Evans, et al. (2004), the growth of the internet has boosted the use of the web for 
teaching and learning. There are several advantages in using a web-based learning system in 
education. It can be used at any time and place, learning materials can easily be updated, 
interaction between the learner and the teacher can be fostered in several ways, multiple media 
such as text, audio, graphics, video and animation can be incorporated, it allows learners to form 
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learning communities, learners’ progress can easily be checked, and a learner-centred approach 
can be implemented to take into account the many differences between learners (Jolliffe, Ritter 
and Stevens, 2001). 

Web-based learning has also been promoted as a better teaching method and as a tool that can 
improve students’ perfonnance, although research findings have shown contradictory results. 
Research has not shown consistent teaching and learning advantages on the use of these 
technologies (Devitt and Palmer, 1999; Greenhalgh, 2001; Watson, 2001). Research in science 
education has also been slow to investigate the outcomes of the changes that ICT brings 
(Greenhalgh, 2001). 

According to Annstrong (2000), many courses are now taught over the internet. Armstrong 
described the use of video conferencing and webcasting technology in traditional courses. He 
claimed that the technology is a sustaining product for higher education because it enables 
lecturers to perfonn many of their existing functions better. Armstrong proposed that classes 
could be taped, allowing the students to review the lecture at any time. This would create the 
opportunity for a student to study the material in a non-linear fashion. Students could leam the 
material in the order that is most comfortable for each individual. He also described how this 
technology allows for guest speakers to make appearances over the webcasting easily and allows 
students to repeat lessons that they do not understand or to learn lessons that they have missed. 

Webcasting technology has made it possible for the delivery of digital information over the 
internet. Webcasting is defined as sending digital infonnation over the internet for reception, 
viewing and listening by the public, possibly involving some interaction between the sender and 
recipients. In a technical sense, webcasting is the published and subscribed or broadcast and tune- 
in methodology for distributing infonnation from one to many recipients (Miles, 1998). This study 
focuses on the effectiveness of using webcasting technology to deliver lectures to students at 
Universiti Putra Malaysia. The research focuses on three modes of webcasting delivery, live 
streaming, pre-recorded and video-on-demand. 

Live Streaming 

Data streaming, commonly seen in the form of audio and video streaming, is when a multimedia 
file can be played back without being completely downloaded first. Most files, like shareware and 
software updates that have been downloaded off the internet, are not streaming data. However, 
certain audio and video files such as Real Audio and QuickTime documents can be streaming 
files, meaning users can watch a video or listen to a sound file while it is being downloaded to 
their computer. With a fast internet connection, users can actually stream live audio or video to 
their computer. Among the advantages of live streaming are (a) transfer of data that allows for 
longer or live video images to be watched as they are downloaded to a computer, (b) streaming 
media that is broadcasted to many people at a set time, and (c) broadcasting (streaming) of 
lectures in a remote location but there is no element of interactivity. 

Pre-recorded 

One of the simplest and most direct applications of webcasting technology is the indexing of pre- 
recorded sessions to bring a level of interactivity to an otherwise linear media. In addition to the 
embedded on-screen menus necessary for time shift-delivered indexed video, cassette jackets can 
carry printed instructions and menus for easy access at any time by the user. It involves recording 
at an earlier time for transmission later and allows delayed broadcasting of lectures at specified 
pre-arranged slots. 
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Video-on-Demand (VOD) 

In video-on-demand (VOD) delivery, videos of lectures are archived on the server for anytime 
anywhere access by students. It involves the ability to stream video and audio content over the 
network in real time to computer with full video control functionality. It is also known as a virtual 
video player for every user. 

Unlike conventional systems, cost effective VOD system provides high quality, full screen digital 
video streaming over an ethemet network using internet protocols. Full on demand control for 
each individual user is available directly to any computer on the network. 

This new technology is being developed all the time, because VOD has so many different 
applications to offer to the customers and its economical possibilities have been seen. Many 
companies, organisations and universities are developing products and standards. The University 
of Putra Malaysia (UPM) is one that is involved in research on the effectiveness of VOD 
technology for students. 


OBJECTIVE OF THE STUDY 

The key objective of the study is to determine whether there are differences in students’ 
achievement through different modes of teaching: face-to-face and three types of webcasting 
technology (live streaming, pre-recorded, and video-on-demand). 

In this study, effectiveness is measured using students’ achievement in a pre-test and post- test 
developed by the course lecturer. 


METHOD 

The study adapted the pre-test post-test control group experimental design to investigate the 
effectiveness of the various modes of webcasting technology in teaching. Figure 1 illustrates the 
research design used. 
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Figure 1. Research design 


The teaching experiment involved a course on Bioinfonnatics, a core course offered for fifth 
semester students of the Bachelor of Science (Biotechnology) program. Apart from the students in 
the fifth semester, there are also students in the sixth and seventh semester. Besides the Bachelor 
of Science (Biotechnology) students, a large group taking this course are students from the 
Bachelor of Science (Biochemistry) program. 

For the teaching experiment, teaching materials amounting to six hours of lectures were prepared 
for the different modes of teaching. The teaching was conducted in three two-hourly sessions. The 
experiment was conducted in the May semester over two years. The first experiment involved 104 
respondents, while the second involved a total of 112 respondents. In the first experiment, 28 
students (27%) were randomly assigned to the Face-to-Face group, 24 students (23%) to the Live 
Streaming group, 26 students (25%) to each of the Pre-Recorded and the VOD groups. In the 
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second experiment, a total of 79 students participated: 21 students (27%) were randomly assigned 
to the Face-to-Face group, 17 students (22%) to the Live Streaming group, 21 students (27%) to 
the Pre-Recorded group, and 20 students (25%) to the VOD group. 

In the first experiment, while students of the face-to-face control group attended the lecture, the 
live streaming group accessed the lecture in a computer laboratory close to the lecture room. The 
students were each provided with a computer equipped with headphones. As in normal lectures, 
the students took notes during the lectures. The next day, the technical group edited the recorded 
lecture and uploaded the lecture in the Webcasting Group server. Apart from storing the lectures 
on the server, the lectures were also prepared in 10 compact discs (CD) as backup. The VOD 
group came in during their free time to view the pre-recorded lectures. They were given the URL 
to download the lecture from the server. The pre-recorded group viewed the lecture at a specified 
time arranged by the technical group in the laboratory. Attendance was recorded for all sessions. 

However, it was observed that students in the VOD and the Pre-recorded groups could have 
shared infonnation with students assigned to the live streaming and face-to-face groups, prior to 
the time that they were to view the lesson. Therefore, in the second experiment, it was decided 
that all four groups would attend the lecture at the same time. Therefore, the lectures were pre- 
recorded without any students attending. The recorded lectures were then edited and merged with 
the Power Point presentation. Considerations were made as to when segments of the lecturer and 
segments of Power Point presentation were to be focused on. The recorded lectures were then 
stored on the server and saved on CD as backup for the teaching sessions. 

During the teaching sessions, the lecturer repeated the lecture with the students from the Face-to- 
Face group and keeping very strictly to the same materials and discussions made during the pre- 
recording of the lectures. At the same time, the live streaming group received the ongoing lecture 
streamed to them live in a computer laboratory. The pre-recorded and the VOD groups viewed the 
pre-recorded lectures from the server. In some cases, where students had problems accessing the 
server, the CD was then used. 

Prior to the teaching experiment, respondents were given a pre-test. The test comprising 60 items 
covered the content to be taught in the six hours of lecture. Post- test was given after the teaching. 
In order to ensure equivalence between pre-test and post-test, the same questions were maintained 
but the sequence of questions was changed and in some of the questions, the questions were 
rephrased. 

In the first experiment, the post- test was given after the six hours of lecture sessions. However, to 
control students’ interaction with lecture materials and course mates or even maturation, the post- 
test in the second experiment was administered after every two-hourly lecture session. This 
ensured that the test measured only what the students had gathered during the two-hour long 
lecture through the different modes of teaching, the face-to-face and the three modes of 
webcasting technology. 

Students were randomly assigned to the four groups in the first experiment. However, in the 
second experiment, extra measures were taken to ensure that the four groups were equivalent. 
Subjects were randomly assigned to groups based on their cumulative grade point average 
(CGPA). Equal distribution of subjects based on gender and race was also considered. Students 
were first grouped into three categories: (a) CGPA greater than or equal to 3.5, (b) CGPA greater 
than 3.0 but less than 3.5 and (c) CGPA greater than 2.5 but less than 3.0. None of the students 
had CGPA less than 2.5. From each category, students were divided into four subgroups taking 
into account gender and race as criteria in forming the subgroups. Each subgroup was randomly 
assigned to form the four groups required for the experiment. The four groups were then randomly 
assigned to the four groups, control (face-to-face) and the three webcasting groups. 
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RESULTS 

Table 1 shows the pre-test and post-test scores of the Phase 1 experiment for all four modes of 
delivery. There is an increase in the post-test scores for all modes of instruction. 


Table 1. Test scores for Experiment Phase 1 


Modes of 

Tests 

Experiment Phase 1 (n=104) 

Delivery 


Mean 

SD 

Face to Face 

Pre-test 

60.89 

10.98 


Post-test 

83.75 

4.00 

Live Streaming 

Pre-test 

62.71 

13.19 


Post-test 

83.13 

4.38 

Pre-recorded 

Pre-test 

60.77 

13.24 


Post-test 

81.15 

4.08 

VOD 

Pre-test 

57.50 

14.02 


Post-test 

80.38 

8.59 

Overall Score 

Pre-test 

60.43 

12.81 


Post-test 

82.12 

5.69 


Table 2 shows the pre-test and post-test scores of the Phase 2 experiment for all four modes of 
delivery. Similar to Phase 1 experiment, there is an increase in the post-test scores for all modes 
of instruction. As noted in the method section, the pre-test and post-test were administered before 
and after every lecture. 

Table 2. Test scores for Experiment Phase 2 

Modes of 
Delivery 

Tests 

Experiment Phase 2 (n=79) 
Mean SD 

Face to Face 

Pre-test 1 

8.81 

1.75 


Post-test 1 

17.38 

2.01 


Pre-test 2 

9.67 

2.08 


Post-test 2 

18.24 

1.41 


Pre-test 3 

9.14 

2.54 


Post-test 3 

18.29 

1.93 


Overall pre-test 

9.21 

2.14 


Overall post-test 

17.97 

1.82 

Live Streaming 

Pre-test 1 

10.94 

2.54 


Post-test 1 

18.00 

1.00 


Pre-test 2 

11.71 

1.74 


Post-test 2 

18.12 

1.45 


Pre-test 3 

9.71 

2.86 


Post-test 3 

17.41 

2.53 


Overall pre-test 

10.78 

2.51 


Overall post-test 

17.84 

1.77 

Pre-recorded 

Pre-test 1 

10.81 

2.62 


Post-test 1 

17.48 

1.60 


Pre-test 2 

11.48 

2.52 


Post-test 2 

18.76 

1.26 


Pre-test 3 

11.00 

2.30 


Post-test 3 

16.81 

1.50 


Overall pre-test 

11.10 

2.46 


Overall post-test 

17.68 

1.65 

VOD 

Pre-test 1 

10.90 

2.00 


Post-test 1 

16.65 

2.37 


Pre-test 2 

11.15 

1.84 


Post-test 2 

18.20 

1.40 


Pre-test 3 

9.40 

1.39 


Post-test 3 

14.70 

2.52 


Overall pre-test 

10.48 

1.90 


Overall post-test 

16.52 

2.56 
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Based on the results of the ANOVA analysis, it was found that there was no significant difference 
between the three pre-test scores and the three post-test scores for each of the four groups. It 
shows that the three pre-test or post-test scores for each group are about the same. 

Since the overall pre-test scores are different between groups, therefore achievement cannot be 
measured based on the subjects’ perfonnance in the post-test. Therefore, comparison was made 
based on the difference in perfonnance between the pre-test and post-test scores for each mode of 
instruction (see Table 3). For example, the increase in performance for Lecture 3, VOD group is 
equal to the difference between Pre-test 3 and Post-test 3 for the VOD group. ANOVA results 
presented in Table 3 show that there is a significant difference between different modes of 
instruction on the mean difference in pre-test scores and post-test scores on the set of tests given 
during the lectures. 


Table 3. ANOVA results on differences in test performance based on modes of lectures 




Sum of Squares 

df 

Mean Square 

F 

Sig 

Increase in performance 

Between Groups 

85.449 

3 

28.483 

3.718 

0.015 

in Lecture 1 

Within Groups 

574.501 

75 

7.660 




Total 

659.949 

78 




Increase in performance 

Between Groups 

48.111 

3 

16.037 

2.860 

0.042 

in Lecture 2 

Within Groups 

420.496 

75 

5.607 




Total 

468.608 

78 




Increase in performance 

Between Groups 

192.410 

3 

64.137 

7.739 

0.000 

in Lecture 3 

Within Groups 

621.539 

75 

8.287 




Total 

813.949 

78 





* significant at 0.05 level 


Based on Lecture 1, post hoc test show that there is a significant difference between the mean 
difference of pre-test and post-test scores between the Face-to-Face group and the VOD group. 
Based on Lecture 3, it was found that there are significant differences between the mean 
difference of pre-test and post- test scores between the VOD group and the Face-to-Face group and 
between the VOD group and the Pre-recorded group. Differences between post test and pre-test is 
found to be greatest for the VOD group (mean = 9.14) as compared to that of the Pre-recorded 
group (mean = 5.81) and the Face-to-Face group (mean = 5.30). Based on these findings, shown in 
Table 4, it was found that the VOD group gained the most from the series of lectures. 

CONCLUSIONS 

The overall results show that the students who went through the VOD group had the greatest gain 
in the tests. Lecture through VOD technology gave positive impact towards students’ learning as 
compared to the other two webcasting technologies used. It could be due to the interactive features 
of VOD technology. Students were able to control the lecture video sequences and suit the 
sequence to their needs. For example, they could decide when to play, pause or stop. As for the 
content, technicians were able to edit the lecture content without changing the originality of the 
lecture. Video lecture editing could increase the audio as well as the visual quality before the 
video was used for subsequent lecture sessions. Editing could rectify the videos before they were 
stored on the server for video lectures filing retrieval. With the data transmission speed of 300 
kilobytes per second (progressive download), the video files played were stable in terms of audio 
and visual. Apart from that, the conduciveness of the room used by the VOD group might also be 
a contributing factor to students’ better scores. 

Among the benefits of VOD are as follows: (a) full video functionality for each user, (b) 
maximum access to content, (c) interactive information delivery, (d) improved security of 
resources, (e) low cost content distribution and (f) full online management interface. In VOD 
delivery, the photographer makes the decision as to when the slides are the focus and when the 
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instructor is the focus. This helps navigate the learning for the students. In a face-to-face setting, 
students may be focusing on the lecturer whereas the slides are the focus at that particular time. In 
all the webcasting modes of lecture, students tend to be very focused. They put the headphones on 
and focus on the computer monitor. However, in face-to-face setting, many things can disrupt the 
students’ attention, especially friends who are sitting close by. 

Table 4. Sheffe’s analysis of mean differences in pre-test and post-test scores of webcasting 


modes 


Difference 

(I) Mode 

(J) Mode 

Mean 

Sig 

95% Confidence 

between 



Difference 


Interval 


Post-test 



(I- J) 


Lower 

Upper 

and Pre-test 





Bound 

Bound 

Lecture 1 

Live Streaming 

Pre Recorded 

0.392 

0.979 

-2.19 

2.97 



Video on Demand 

-1.513 

0.428 

-4.10 

1.07 



Face to Face 

1.309 

0.564 

-1.30 

3.92 


Pre Recorded 

Live Streaming 

-0.392 

0.979 

-2.97 

2.19 



Video on Demand 

-1.905 

0.183 

-4.35 

0.54 



Face to Face 

0.917 

0.772 

-1.56 

3.39 


Video on Demand 

Live Streaming 

1.513 

0.428 

-1.07 

4.10 



Pre Recorded 

1.905 

0.183 

-0.54 

4.35 



Face to Face 

2.821* 

0.018 

0.35 

5.29 


Face to Face 

Live Streaming 

-1.309 

0.564 

-3.92 

1.30 



Pre Recorded 

-0.917 

0.772 

-3.39 

1.56 



Video on Demand 

-2.821* 

0.018 

-5.29 

-0.35 

Lecture 2 

Live Streaming 

Pre Recorded 

-0.874 

0.734 

-3.08 

1.34 



Video on Demand 

-2.160 

0.058 

-4.37 

0.05 



Face to Face 

-0.638 

0.880 

-2.87 

1.60 


Pre Recorded 

Live Streaming 

0.874 

0.734 

-1.34 

3.08 



Video on Demand 

-1.286 

0.383 

-3.38 

0.80 



Face to Face 

0.236 

0.992 

-1.88 

2.35 


Video on Demand 

Live Streaming 

2.160 

0.058 

-0.05 

4.37 



Pre Recorded 

1.286 

0.383 

-0.80 

3.38 



Face to Face 

1.521 

0.247 

-0.59 

3.64 


Face to Face 

Live Streaming 

0.638 

0.880 

-1.60 

2.87 



Pre Recorded 

-0.236 

0.992 

-2.35 

1.88 



Video on Demand 

-1.521 

0.247 

-3.64 

0.59 

Lecture 3 

Live Streaming 

Pre Recorded 

1.896 

0.262 

-0.79 

4.58 



Video on Demand 

-1.437 

0.509 

-4.12 

1.25 



Face to Face 

2.406 

0.102 

-0.31 

5.12 


Pre Recorded 

Live Streaming 

-1.896 

0.262 

-4.58 

0.79 



Video on Demand 

-3.333* 

0.005 

-5.87 

-0.79 



Face to Face 

0.510 

0.956 

-2.06 

3.08 


Video on Demand 

Live Streaming 

1.437 

0.509 

-1.25 

4.12 



Pre Recorded 

3.333* 

0.005 

0.79 

5.87 



Face to Face 

3.843* 

0.001 

1.27 

6.42 


Face to Face 

Live Streaming 

-2.406 

0.102 

-5.12 

0.31 



Pre Recorded 

0.510 

0.956 

-3.08 

2.06 



Video on Demand 

-3.843* 

0.001 

-6.42 

-1.27 


* The mean difference is significant at 0.05 level 


The results of the data analysis also show that the effectiveness of Face To Face lecture is lowest 
as compared to the use of webcasting technologies. Apart fonn students’ concentration being 
disrupted by friends and whatever other happenings during the lecture session, students’ 
concentration may be also be distracted because the recording was done during their session. The 
recording done during the face to face lecture is streamed live to the Live Streaming group. 
Therefore, such existing logistics problems need to be resolved to reduce interference, such as 
providing the accessibility of a centre lecture room where the camera can be remotely controlled 
by a cameraman from a control room. The settings in the other webcasting modes are quite 
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different from the VOD mode. Headphone usage cuts off the connection between students and the 
existing environment and thus further reduces the interference, allowing students to give full 
concentration to the lectures. 

Perfonnance of the pre-recorded group is also low as compared to the VOD and Live Streaming 
groups. The production of the pre-recorded materials is based on an existing video, which needs 
to be encoded. The encoding process can reduce the video quality. Video that is encoded can be 
streamed at a compression rate of 150 kilobytes per second. Students who receive video at a lower 
compression rate than 150 kilobytes per second have a poorer video quality. Due to this problem, 
students’ concentration towards the pre-recorded lectures can be lower. 

Live streaming is being streamed through progressive download with a compression rate of 300 
kilobytes per second. Since the broad band loading burden is high, this reduces the data 
transmission speed rate to about 150 to 200 kilobytes per second. It also reduces the satisfaction of 
student learning due to the poorer quality of audio and visual. 

In summary, VOD can be used as an alternative to face to face lectures. This study shows the 
effectiveness of VOD mode of delivery as compared to pre-recorded and live streaming 
webcasting deliveries and face to face delivery. However, more studies need to be conducted, 
especially to ascertain the effectiveness of Webcasting technology in the delivery of lectures for 
content such as mathematics, and for courses that involve demonstration or laboratory work. 
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